The increased power and speed of contemporary computer systems has resulted in a corresponding demand for significantly increased mass storage capability because of the increased volumes of data dealt with by the systems and the increased size of the operating system and applications programs executed by such systems. There are a variety of mass storage devices available, such as disk drives, optical drives and tape drives, and one approach of the prior art to the mass storage problem is the use of mass storage systems associated with the computer systems. An example of a mass storage system is described in U.S. Pat. No. 5,257,367, cited above, and may be taken as illustrative of this general approach to the mass storage problem. Typically, such a mass storage system will include a plurality of relatively fast mass storage devices, such as disk drives, together with one or more slower but larger archival type storage devices, such as a tape drive or optical drive, and a processor based storage system management unit, sometimes referred to as a "disk platform", to manage and control the transfer and storage of data between the mass storage system and the associated host computer system. In many such systems, such as that described in U.S. Pat. No. 5,257,367, the storage system management unit is connected to the host computer system in the same manner as a conventional disk drive, and through a conventional disk drive controller normally present in the host computer system, to appear to the host computer system as a conventional disk drive, although with significantly increased storage space.
A recurring problem with data storage, however, and increasingly so as the capacity of mass storage systems increases, is that of protecting the data from loss, for example, due to failure of a disk drive in a mass storage system, and a number of techniques have evolved in the prior art for protecting data in mass storage systems. One technique, for example, is the use of parity information, which is essentially an additional body of data that is generated from the stored data, usually as the data is being stored, and that contains information that may be used to reconstruct or correct lost or damaged data.
Another technique is "striping", which is customarily employed as a technique for increasing the speed with data may be written to or read from the disk drives of a mass storage system by distributing a body of data across a plurality of disk drives so that reads and writes from and to the disk drives with respect to segments of the data can be overlapped. Striping also serves to protect the data, in which role it is usually used together with parity, by storing of a body of data across a plurality of disk drives so that any data that is destroyed or damaged on one of the disk drives may be reconstructed from the remaining data and the parity information. An extreme embodiment of this technique is found, for example, in the Thinking machines Corporation CM-2 system which operated with 39 bit words, each containing 32 data bits and 7 parity bits, and stored the bits of each word in parallel across 39 disk drives, with one bit of each word being stored on each drive.
A third commonly used technique of the prior art, which is again often used with parity data, is "mirroring" wherein the entire body of data and parity information stored in the mass storage system is duplicated on a second set of disk drives, which may reside in a second mass storage system.
These techniques for data protection of the prior art, however, have not been entirely satisfactory for a number of reasons. For example, the combination of striping with parity as implemented in most mass storage systems will generally provide adequate protection in the event a failure of a single disk drive in a mass storage system, but will generally not allow data reconstruction or recovery if two or more disks fail. Also, while it is possible to construct mass storage systems using striping and parity that will allow recovery from multiple disk drive failures, this approach requires either increasing the granularity of data storage, that is, providing more disk drives so that smaller blocks of data are stored on each drive, or significantly increasing the amount of parity information, again requiring more disk storage space, or both. As a consequence, the cost of the mass storage unit soon becomes impractical and the performance of the mass storage system, that is, the rate at which data may be written and read, may suffer.
Mirroring the entire body of data parity information stored in a mass storage system, in turn, immediately doubles the number of disk drives required to store the data and parity information and, again, the cost of the mass storage unit soon becomes impractical and the performance of the mass storage system, that is, the rate at which data may be written and read, may suffer.
Yet another problem with mass storage systems of the prior art is that the generation of parity information and the writing of the parity information into the disk drives significantly increases the time required to store incoming data in the mass storage system. The additional time required to generate and store parity information may, in turn, be unacceptable in certain circumstances and may actually result in a failure to acquire and store the data, for example, when large volumes of data are being received at short intervals or when the data is available only for limited periods, as when data is being downloaded from satellite a link.
The present invention provides a solution to these and other problems of the prior art.